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GUTOANCE OF INVASIVE MEDIC AL DEVICES BY 
WIDE VIEW THREE DIMENSIONAL ULTRASONIC IMAGING 

This invention relates to ultrasonic diagnostic imaging and» more 

5 particularly, to the use of three dimensional ultrasonic diagnostic imaging to guide the 
placement and operation of invasive (interventional) medical devices in the body. 

Ultrasonic infiaging is commonly used to image the insertion, use or 
operation of medical devices and instniments within tiie body. One such conimioh use 
of ultrasound imaging is in the conduct of a biopsy procedure. An ultrasound probe is 

10 used to image the pathology of interest for tiie procedure such as a suspected tumor or 
cyst The probe is manipulated until the patholo^ is visible in the image plane. A 
biop^ needle attachment for the probe tiien guides the insertion of the biopsy needle 
within tiie image plane and toward the pathology. The clinician follows the travel of the 
needle in the ultrasound image, being careful to keep the probe stationary and the needle 

IS within the image plane until the needle tip reaches tiie pathology. A specimen is 
extracted through the needle and the needle is withdrawn fiom the body. Ultrasonic 
imaging is thus used to guide the travel of tiie needle into the body and to observe tiie 
conduct of tiie biopsy procedure. 

I Biopsy needles have been designed with their own ultrasonic transmitters 

20 or receivers which interact with the imaging probe. Such ultrasonically responsive 
needles allow the needle and the imaging probe to signal each other and enable the 
needle and its tip to be more clearly identified in the ultrasound image plane. 
Ultrasonically responsive biopsy needles are described in US Pat 5,158,088, for 
instance. 

25 The planar imaging techniques are limited in that they provide a 

restricted, single image view of the internal site of the procedure. It would be desirable 
to provide a greater field of view of the site of the procedure to enable the clinician or 
surgeon to better guide and conduct the procedure. Improved imaging would assist 
biopsy procedures and also facilitate a wide range of invasive procedures such as the 

30 placement of stents and cannulae, the dilation or resection of vessels, treatments 
involving the heating or fireezing of internal tissues, the placement of radioactive seeds 
or prosthetic devices such as valves and rings, the guidance of wires or catheters 
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fhiough vessels for the placement of devices such as pacemakers, implantable 
caidiovertois/defibrillatois, electrodes, and guide wires, the placement of sutures, 
staples and chemical/gene sensing electrodes, the guidance or operation of robotic 
surgical devices, and the guidance of endoscopic or minimally invasive surgical 

S procedures. Ultrasonic guidance would thus find expanded use in a broad range of 
invasive or intervmtional clinical applications including cardiac, pulmonary, central and 
peripheral nervous system procedures, gastrointestinal, musculoskeletal, gynecological, 
obstetrical, urological, ophthalmologic and otorhinolaiygologic procedures. 

In accordance with the principles of the present invention, three 

10 dimensional ultrasonic imaging is used to guide or monitor the conduct of the 
placement and/or use of invasive (interventional) medical devices such as those 
enumerated above. In one embodiment the location of the interventional device or its 
activities are recorded in a three dimensional ultrasound image which consolidates 
information from both the three dimensional ultrasonic imaging system and the 

15 interventional system. The consolidated image may be viewed on the ultrasound 
system, on the interventional syst^, or on the display of a combined ultrasonic imaging 
and interventional device system. The interventional device is shown in a detailed 
ultrasonic image and the balance of the volumetric region in which the device is located 
is shown in a wide field of view. The detailed and wide fields of view may be displayed 

20 separately or ov^lapping in spatial aligmnent on an image display. The wide field of 
view may be shown in two or three dimensions. In accordance with a fiirther aspect of 
the present invention the locus of the interventional device is ultrasonically scanned in 
greater detail than the surrounding volume for greater visual precision and higher firame 
rate of display of the guidance or use of the interventional device. 

25 In the drawings: 

FIGURE 1 illustrates in block diagram form the use of three dimensional 
ultrasonic imaging to guide or monitor an invasive instrument and procedure. 

FIGURE 2 illustrates the three dimensional ultrasonic imaging of a 
catheter in the heart by a transthoracic transducer probe. 

30 FIGURE 3 illustrates in block diagram form the functional subsystems of 

a three dimensional ultrasonic imaging system suitable for use in an embodiment of the 
present invention. 
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FIGURB 4 illustrates in block diagram foim another embodiment of tiie 
use of three dimensional ultrasonic imaging to guide or monitor an invasive instrument 
and procedure. 

FIGURE 5 illustrates a method for positipnally locating an invasive 
5 medical device within the body by means of a two dimensional array transducer. 

FIGURE 6 illustrates a second method for positionally locating an 
invasive medical device within Ifae body. 

FIGURB 7 illustrates flie scanning of the volume around an invasive 
device with greater beam density than the surrounding image volume. 
10 FIGURES 8-11 illustrate ultrasound displays of a volume of interest 

together with a greater volumetric field of view containing the volume of interest. 

FIGURE 12 illustrates the display of three dimensional, two 
dimensional, and quantified ultrasonic measures of an interventional site. 

FIGURE 13 illustrates the display of a detailed three dimensional 
1 5 ultrasonic image of an interventional device along with a greater volumetric view of the 
location of the mterventional device. 

FIGURE 14 illustrates the recording of the locus of interventional 
procedures in a three dimensional ultrasonic image. 

FIGURE 15 illustrates the recording of the loci of interventional 
20 procedures in a wire frame model derived from three dimensional ultrasonic image data. 

FIGURE 16 illustrates a simultaneous view of a live three dimensional 
image of an interventional device, a wire frame model recording the loci of 
interventional procedures, and EGG waveforms relating to the loci: 

FIGURES 17-21 are flowcharts illustrating methods for combining 
25 image and/or locational data from a three dimensional ultrasonic imaging system and an 

interventional system. 

FIGURE 22 illustrates in block diagram form a consolidated system for 
the conduct of an invasive procedure assisted by three dimensional ultrasonic imaging. 

Referring first to FIGURE 1, the use of three dimensional ultrasonic 
30 imaging to guide or monitor an invasive instrument and procedure is shown in partial 
block diagram form. On the left side of the drawing is a three dimensional (3D) 
ultrasonic unaging system including a probe 10 having a two dimensional array 
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transducer. The transducer array transmits ultrasonic beams over a volumetric field of 
view 120 under control of an ultrasound acquisition subsystem 12 and receives echoes 
in response to the transmitted beams which axe coupled to and processed by the 
acquisition subsystem. The echoes received by the elements of the transducer array are 

S combined into coherent echo signals by the acquisition subsystem and the echo signals 
along Willi ttie coordinates fiom which fliey are received (r,6,q) for a radial transmission 
pattern) are coupled to a 3D unage processor 14. The 3D image processor processes the 
echo signals into a three dimensional ultrasonic image which is displayed on a display 
1 8. The ultrasound system is controlled by a control panel 16 by which tihe user defines 

10 the characteristics of the imaging to be performed. 

Also shown in FIGURE 1 is an interventional device system. The 
intervmitional device system includes an invasive (interventional) device 30 which 
performs a function within the body. In this drawing the interventional device is shown 
as a calbeter, but it could also be some other tool or instrument such as a needle, a 

IS surgical tool such as a dissection instrument or stapler or a stent delivery, 
electrophysiology, .or balloon catheter, a therapy device such as a high intensity 
ultrasound probe or a pacemaker or defibrillator lead, a diagnostic or measurement 
device such as an IVUS or optical catheter or sensor, or any other device which is 
manipulated and operates within the body. The interventional device 30 is manipulated 

20 by a guidance subsystem 22 which may mechanically assist die maneuvering and 
placement of the interventional device within the body. The interventional device 30 is 
operated to perform its desired function such as placing an item at a desired location, or 
measuring, illuminating, heating, freezing, or cutting tissue under the control of an 
interventional subsystem 20. The interventional subsystem 20 also received 

25 information from the interventional device on the procedure being performed, such as 
optical or acoustic image information, temperature, electrophysiologic, or other 
measured information, or information signaling the completion of an invasive operation. 
Information which is susceptible of processing for display is coupled to a display 
processor 26. The interventional device may also have an active position sensor 32 

30 which is used to provide information as to the location of the working tip within the 
body. The active position sensor 32 may operate by transmitting or receiving signals in 
the acoustic, optical, radio frequency or electromagnetic spectrum and its output is 
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coupled to a device position measurement subsystem 24. Altemately the sensor 32 may 
be a passive device such as a difiBraction grating which is highly reflective of ultrasonic 
energy transmitted by the piobe 10. Position information of the interventional device is 
coupled to the display processor 26 when appropriate for the processing or display of 
5 information concerning the position ofthe interventional within the body. Information 
pertinent to the fimctioning or operation of the interventional device is displayed on a 
display 28. The interventional device system is operated by a user through a control 
panel 27. 

In the embodiment of FIGURE 1 the invasive device 30 is shown as a 

10 catheter which is performing a function at the wall of the left ventricle 102 ofthe heart 
100. The full extent of the endocardial wall of tiie left ventricle is visible by three 
dimensional ultrasonic imaging of flie volumetric field of view 120 of flie 3D ultrasonic 
imagmg system. Tlie working tip ofthe interventional device 30 may include an x-ray, 
r.f. or ultrasonic device for imaging or ranging the endocardium, or a physiologic or 

IS thermal sensor which conducts diagnostic measurements of the endocardium, or an 
ablation device which treats lesions on ihe endocardium, or a placement device for an 
electrode, for example. The tip of the interventional device is manipulated to a point on 
the heart wall where such a function is to be performed by operation of the guidance 
subsystem 22. The interventional device is then commanded to perform its intended 

20 procedure by the mterventional subsystem 20, and the location at which the-procedure is 
performed by the device position measurement subsystem 24 which receives or 
transmits a signal from ftie sensor 32 at the time of the procedure, for instance. 

The invasive procedure may be assisted by monitoring the procedure 
simply by visualizing the site of the procedure, tiie wall of the left ventricle in the 

25 foregoing example, by use of the three dunensional ultrasound ^stem. As the 
interventional device 30 is manipulated within the body the three dimensional 
environment in which the device is operated can be visualized in three dimensions, 
enabling the operator to anticipate turns and bends of orifices and vessels in the body 
and to precisely place the working tip ofthe interventional device at the desired site of 

30 the procedure. It is necessary to see a large field of view in order to provide gross 
navigation \vith enough detailed resolution to guide the intervention within the vicmity 
of the invasive device. The operator can maneuver and reposition the probe 10 to 
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constantly keep the interventional device 30 within the probers volumetric field of view. 
Since in the piefened embodiment fhe probe 10 has a two dimensional array which 
rapidly transmits and receives electronically steered beams, rather than a mechanically 
swept transducer, real time three dimensional ultrasonic imaging can be performed and 
S die interventional device and its procedure observed continuously and precisely in three 
dimensions. 

hi accordance with a finiher aspect of this first embodiment of the 
present invention, a signal path 42 connects the ultrasound acquisition subsystem 12 of 
the ultrasound system and fhe device position measurement subsystem 24 of the 

10 interventional device system to allow synchronization of the imaging system and the 
• interventional device. This synchronization allows image acquisition and interventional 
device operation to be done at different time interleaved intervals if tiie operation of one 
device would interfere with fhe other. For example, if fhe interventional device 30 is 
perfonning acoustic imaging of the heart or vessel wall, it is desirable for these acoustic 

15 intervals to occur when they will not be disrupted by acoustic transmissions fix>m the 
3D imaging probe 10. It may be desirable to suspend imaging when the interventional 
device is transmitting high energy signals for ablation or some other procedure that 
would interfere with the imaging signals from flie probe 10. The synchronization also 
enables the ultrasound system to ask for and receive position information from fhe 

20 interventional device when the ultrasound system is producing a 3D ultrasound image 
with an enhanced representation of the position of the interventional device shown in 
the 3D ultrasound image. The ultrasound system may also ask for and receive position 
information from the interventional device when the ultrasound system is recording the 
location of a procedure performed by the interventional device for future reference, as 

25 will be discussed more fiilly below. 

In accordance with a another aspect of this first embodiment of the 
present invention, image data may be exchanged over a signal path 44 between the 3D 
ima^ processor 14 of the ultrasound system and the display processor 26 of the 
interventional device system for tibte formation of a 3D image containing information 

30 from bofli systems. This enables the display of an image of fhe interventional device 
30, produced by fhe interventional device system, as part of a 3D ultrasound image 
produced by the ultrasound system. Such a fusion of the imaging capabiUties of both 
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systems better enables the physician to guide and utilize the interventional device, aided 
by the extensive three dimensional field of view afforded by tiie ultrasomxd system and 
the device image data produced by the interventional device system. 

FIGURE 2 illustrates practice of the present invention when the three 
dimensional ultrasound probe used is a transthomcic probe 10. In this example the 
heart 100, shown in partial outfine behind the rib cage 110,112, is located behind the 
left side of the rib cage. Outlined within tiie heart and cross-hatohed is the left ventricle 
102 of the heart 100. Hie left ventricle can be accessed for ultrasonic imaging by 
scanning the heart from between the ribs 110,112 for adult patients and, for some 
pediatric patients, by scaiming upward from below Ifae lowest rib 112. The probe 10 
scans the heart from the heart apex 104 as indicated by the outline 120 of &e volumetric 
field of view scanned by flie probe 10. As FIGURE 2 illustrates, the left ventricle 102 
can be ftilly encompasses and scanned by the volumetric field of view directed from 
between the rib cage 110,112. 

While this embodiment illustrates phased array scanning of tiie 
volumetric region 120 in a conical field of view, one skilled in the art will recognize 
that other scan formats may also be employed such as those which scan a rectangular or 
hexagonal pyramidal field of view. It will also be ^ypreciated that probes other ttian 
transthoracic probes may be used for three dimensional scanning such as 
transesophageal probes, intracavity probes such as vaginal or rectal probes, and 
intervascular probes such as caflieter-moimted transducer probes. While an 
electronically scanned two dimensional array transduce is preferred, mechanically 
scanned arrays may be preferred for some applications such as abdominal procedures. 

FIGURE 3 illustrates some of the components of the 3D ultrasound 
system of FIGURE 1 in further detail. The elements of a two dimensional array 
transducer 50 are coupled to a plurality of microbeamformers 62. The 
microbeamformers control the transmission of ultrasound by the elements of the array 
transducer 50 and partially beamform echoes returned to groups of the elements. The 
microbeamformers 62 are preferably fabricated in integrated circuit form and located in 
the housing of the probe 10 near the array transducer. Microbeamformers, or subarray 
beamformers as they are often called, are more fully described in US Pats. 6,375,617 
and 5,997,479. The probe 10 may also include a position sensor 52 which provides 
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signals indicative of the position of the probe 10 to a ttansducer position detector 54. 
Hie sensor 52 may be a magnetic, electroinagnetic, radio fiequency, infrared, or o4er 
type of sensor such as one which transmits a sign^ that is detected by a voltage 
impedance circuit The transducer position signal 56 produced by the detector 54 may 
be used by ttie ultrasound system or coupled to the interventional device system when 
usefol for file formation of spatially coordinated imagps contaming information from 
both systems. 

The partially beamfomed signals produced by fte microbeamformers 62 
are coupled to a beamformer 64 where fbs beamformation process is completed. The 
resultant coherent echo signals along the beams are processed by filtering, ampHtude 
detection. Doppler signal detection, and other processes by a signal processor 66. The 
echo signals are then processed into image signals in «ie coordinate system of the probe 
(r,e.<p for example) by an image processor 68. The image signals are converted to a 
desired image fonnat (x,y,z Cartesian coordinates, for example) by a scan converter 70. 
15 The three dimensional image data is coupled to a volume renderer 72 which renders a 
three dimensional view of the volumetric region 120 as seen fiom a selected look 
direction. Votame rendering is well known in the art and is described in US Pat 
5,474,073. Volume rendering may also be performed on image data which has not been 
scan converted as described in US [patent application serial number 10/026,996, filed 
20 December 19. 2001 by AUstair Dow and Paul Detmer.] During two dimensional 
imaging the image plane data bypasses the volume renderer and is coupled directly to a 
video processor 76 which produces video drive signals compatible wiA the 
requirements of the display 18, The volume rendered 3D images are also coupled to the 
video processor 76 for display. The system can display individual volume rendered 
25 images or a series of volume rendered images showing the dynamic flow and motion of 
■ the anatomy being imaged in real time, hi addition, two volume renderings can be done 
of a volumetric data set fiom sUghtly offset look directions, and the two displayed 
simultaneously on a stereoscopic display as described in US Pat [appUcation serial 
number 60/43,096 (attorney docket 020478), filed December 3, 2002 by Jonathan Ziel 
30 and entitled '"Method and Apparatus to Display 3D Rendered Ultrasound Data on an 
Ultrasound Cart in Stereovision"]. A graphics processor 74 receives either scan 
converted image data fiom the scan converter 70 or unscan-converted image data from 
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fhe image processor 68 for analysis and the generation of graphics, such as visual 
emphasis of ttie tip of an interventional device or the detection of the border of an organ 
within flie image field. The visual emphasis may be provided by an enhanced or unique 
brightness, color, or volume rendering process for imaging the tip of «ie device, for 
5 example. iTie resultant graphics are coupled to Ae video processor where Ihey are 

coordinated and overlaid wilh the image for display. 

FIGURE 4 illustrates another embodiment of Ihe present invention. This 

embodiment differs ftom that of HGURE 1 m tiiat there is a connection 46 fiom the 
probe 10 to the device position measurement subsystem 24 of the mterventional device 
10 system, fa ttiis embodiment the probe 10 includes the position sensor 52. Rather tiian 
process the probe position signal by the transducer position detector 54 m flie ultrasound 
system, the signal is processed by the same subsystem that processes fbs position signal 
ftom flie interventional device 30. The two position signals allow a direct correlation of 
the positions of tiie interventional device and flie probe to be performed by tiie 
15 interventional device system and used for coordinated imaging offte two systems. 

FIGURE 5. illustrates one technique by which the position of tiie 
interventional device 30 is detected by flie probe 10. fa the drawing a transducer 32 
located on the interventional device 30 transmits an acoustic pulse which is received by 
three elements 51, 51' and 51" of tiie transducer array 50. The elements 51 and 51' are 
spaced apart ftom element 51" by known distances a and b. By measuring tiie times of 
arrival of fhe pulse from the interventional device at the tiiree elements flie position of 
fhe transducer 32 witii respect to tiie array transducer 50 can be calculated by 
triangulationl This can be performed by computing 

ia+v(tl-d)/ 

x_po5ition = ^ " /2a 

y_posmon = ^ 



20 



z_posiHon = ^ytl -x_ position -y_ position 

25 where t6 is flie time of flight of flie pulse to element 51", ta is flie time of flight to 
element 51 and t*. is flie time of flight to element 51' and v is flie speed of sound 
(approximately 1550 m/sec) in flie body. 
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FIGURE 6 illustrates a technique for locating the position of the 
interventional device in which ultrasound pulses from the imaging probe 10 are 
received by a transducer 32 on the interventional device 30. The transducer 32 listens 
for pulses from the probe 10. The transmit beam wift the strongest signal or the 
5 shortest time of flight to tiie transducer 32 corresponds to the direction of tiie transducer 
32 with respect Id flie transducer array 50. The distance R between the probe 10 and the 
transducer 32 is determined by the time of ffi^t of tiie transmitted pulse. In fliis 
embodiment the connection 46 or the synchronizing line 42 would be usbd to exchange 
the times of transmission and/pr reception infimnation between the ultrasound system 
10 and tiie interventional device system. 

An interventional device with its own ultrasonic transducer can be used 
to provide other locational information. For instance, if the interventional device has an 
ultrasonic transducer which is capable of transmitting and receiving from a distal end of 
the device, flie transducer can be used for ranging, sending out pulses and receiving 
15 echoes from targeted tissues or tissue interfaces and tiiereby monitoring or measuring or 
displaying tiie distance between a distal part of the device and nearby anatomy from the 
time-of-flight of the transmit-receive interval. In one embodiment of such a device, the 
operator can visually observe the device approaching the tissue of interest inlhe three 
dimensional image, and can also observe a measure of the distance between tiie device 
20 and flie tissue. The distance measure can for example be displayed numerically in 
centimeters, or in a quantified display such as an M-mode display, m which the 
progressive closure of the device with the tissue can be seen over an interval of time. 

The location of the interventional device may also be detected if desired 
by signal and/or image processing techniques in the ultrasound system. For example, 
25 tiie use of a highly reflective element on the interventional device 30 such as a 
dif&action grating as shown in US Pat 4,401,124 may be used to produce a distinctly 
identifiable echo return from tiie interventional device. Alternatively if tiie 
interventional device has excessive specular reflective characteristics, it may be 
desirable to use a device which provides better scattering and/or absorption of 
30 ultaasound energy, such as one with a roughened or absorptive surface. This would 
allow flie system gain to be increased for better definition of the tissue structure in tiie 
3D image, while at tiie same time tiie interventional device does not overwhelm tiie 
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iinagewithbright.stiongIyieflectedechoes. Another alternative is to identify the shape 
of liie interventional device tip by image processing such as automated border detection 
of the tip in Ihe images. Yet anoflier approach is to cause the tip of Ae interventional 
device to be vibrated so fliat it produces a Doppler return as described in US Pat 
5 5,095,910. Other embodiments using transducers on the interventional device may be 
found m US Pats. 5,158,088 and 5.259,837. Other sensor types for invasive devices 
such as magnetic field coils may be employed as described in US Pat 6,332,089. 

One of the difBculties when conducting 3D ultrasonic imaging of 
interventional devices is that a wide or deep volumetric field of view is .usually required 
10 to adequately image the device m the environs of its pafli of travel and procedure. This 
means fliat a significant number of beams must be transmitted and received to 
adequately scan the volmnetric field of view with the desired resotation and without 
spatial aliasing. Furthermore, the deplh of the field of view requires significant times of 
travel of the transmitted pulses and received echoes. Hiese characteristics cannot be 
15 avoided as they are mandated by the law of physics governing the speed of sound m the 
body. Accordingly a significant amount of time is needed to acquire a full floee 
dunensional image, and the frame rate of display will often be lower than desired. 
FIGURE 7 illustrates one solution to this dilemma. wHch is to use a different scanning 
■ methodology in the vicinity of the interventional device 30 than is employed in the outer 
20 reaches of the wide field of view, fa the illustrated example, Ae location of the 
interventional device is determmed by one of the techniques discussed above. Hus 
information is communicated to the ultrasound acquisition subsystem 12, when then 
transmits a greater beam density in the volumetric region 122 surrounding the 
interventional device 30. In the remainder of the volumetric field of view 120 more 
25 widely spaced transmit beams are employed. The space between the widely spaced 
transmit beams may be filled in by interpolating synAetic echo signals if desired. By 
this technique fee volumetric region 122 surrounding the interventional probe will be 
shown with higher definition and resolution, enabling the physician to accurately guide 
and use the device at the site of the procedure. The remaining volmnetric space wiU be 
30 shown with less definition but sufficient to orient the interventional device and 
procedure in the surrounding tissue. The beam density within the volumetric region of 
the interventional device can be uniformly high with the surromiding space scamied 
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with unifonnly lesser beam density. Alternatively, fte highest beam density can be 
employed in fhe vicinity of fhe device sensor 32, wiHi Ae beam density declining wifh 
greater distances fiom flie interventional device. By continuously tracking the location 
of the interventional device 30 fhe volumetric region 122 is constantly redefined as 
5 needed to spatially correspond to the location ofthe interventional device 30. 

Another variation in beam density which may be enq)loyed is to use 
different orders of received multilines m the proximity of the interventional device and 
in the surrounding volume. Variation in multiline order and transmit beam density can 
be used together or separately. For example the same spatial transmit beam density can 
10 be used throughout the scanned volume, but higher order multiline (a greater number of 
differently steered receive Imes for each transmit beam) is used in the vicinity of fte 
interventional device than in fhe surrounding volmne to produce a more detailed image 
intheregionoftheinterventionaldevice. As a second example, a lesser spatial transmit 
beam density is used in the surrounding volume than is used in 4e vicinity ofthe 
15 interventional device. Higher order multiline is then used inihe surrounding volume to 
fin in the spaces between transmit beam axes with a relatively largp number of received 
multilines and a lower order of multiline reception is used in fhe vicinity of flie 
interventional device to fill in the lesser spacing between transmit beam axes wifh a 
relatively fewer number of received multilines fijr each transmit beam. This latter 
20 approach will reduce the multiline artifact of echo intensity variation as a fimction of 
the distance of each received multiline from the transmit beam axis in the vicinity ofthe 
interventional device where a more detailed view is desked. 

FIGURES 8-13 illustrate displays 99 of olher techniques for dealing wife 
this problem of 3D frame rate decline. In FIGURE 8 the 3D locus 82 of the 
25 interventional device 30 is scanned with a high beam density and/or fiame rate. Hie 
volumetric region 82 is shown in the same display 99 with a wider planar field of view 
80. Hie plane 80 may be scanned less frequently or wifh a lesser beam density than ftat 
ofthe volumetric region 82 ofthe device 30. Different orders of multiline reception 
may also be employed. Scanning fhe area 80 with a lesser beam density means Aat a 
30 greater volume or area can be scanned with fhe same number of transmit-ieceive cycles 
as were needed to scan the higher beam density volume 82. Thus a wider field of view 
can be scanned with lesser beam density and the frame rate of fhe fiiU image shown in 
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HGURE 8 is increased. An outline 82' depicts the location of the interventional device 
volume 82 in relation to ihe image plane 80. In FIGURE 9 Ae wider field of view is 
provided liy a 3D image 84 with a lesser beam density and/or frame rate than that of flie 

separate interventional device volume 82. The location of the interventional device 
5 volume82inrelationtoAegreatervohimehnage84isindicatedl>ylheoutUne8r. In 

nOURE 10 the interventional device volume 82 is shown in its correct spatial location 
in relation to flie image plane 80. In HGURE 11 the interventional device volume 82 . is 
shown in its true spatial position in the more lightly sampled volmne 84. the approach 
depicted in FIGURE 7. 

10 FIGURE 12 illustrates a display 99 of a three dimensional ultrasonic 

image 400.of a system of vasculature 404.406. An interventional procedure such as the 
placement of a stent has been performed at a location 410 that is recorded in the image. 
Ihe location 410 was marked by detecting the location of the interventional device at 
the time of the setting of the stent and is thereafter continuously shown in its recorded 
15 spatial location in the body. Ihe three dhnensional image 400 provides a 
comprehensive view of the locus of the stent, and is produced wifli a lower beam 
density and/or frame rate than that of a plane 402 of the volumetric region 400. The 
image of the plane 402 is shown adjacent to the votometric field of view and contains 
the locational marker 410. By scamiing this image plane 402 wiA a greater beam 
20 density and/or a higher frame rate the physician is better able to observe the result of the 
interventional procedure, to addition, an image 408 particular to &e procedural site 
location 410 is shown in the display. This will typically be a time4«ised display such as 
an EKG trace, a spectral Doppler display, an M-niode display, or a color M-mode 
display, aU of which provide physiological mformation as a function of time. 
25 FIGURE 13 illustrates a display 99 employing the techniques shown in 

FIGURES 9 and 1 1. to this display a wide field of view three dhnensional hexagonal 
image 300 which is scamied at a lesser beam density and/or fiame rate reveals the 
expanse of a vasculature system 12. An interventional device 12 performing a 
procedure at a point m the vasculature is shown in a more highly resolved volumetric 
30 region 306. The mterventional device volume 306 is also shown separately m the same 
display in a zoomed view 308 m which the point of the procedure is shown in greater 
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detail. This embodiment combines sevend of Ihefeatifles of IheeariiCT 
Other variations will readily occur to those skilled in the art 

FIGtJRE 14 iUustrates an embodiment of flie present invention in vrbich 
the results of an invasive treatment regime are recorded in a three dimensional 
5 ultrasonic image. This drawing illustrates a catheter 30 with a working tip which 
ablates lesions on the heart wall. As each lesion is treated the location of the treatment 
is marked by use of the position sensor 32 or one of flie interventional device detection 
techniques discussed above. Another technique for detecting the position of die caflieter 
is to apply electrodes of different voltages to opposite sides of flie body to create a 
10 spatial voltage gradient across the bo^. Hie sensor 32 detects the impedance at its . 
locatioii in the gradient field, which corresponds to the spatial location of the catiieter 
working tip. By using multiple electrodes at different times and places on the body, the 
location of the working tip can be sensed in three dimensions. These treatment 
locations are stored in memory and can be used to map the general region where the 
15 treatment occurred, such as the heart wall, to accordance with a further aspect of flie 
present invention, the treatment location information is merged with the three 
dimensional ultrasonic image data to visually mark the treatment locations on the 
endocardial waU as shown by the circular markers 92,94 inHGURE 14. Recordmgfhe 
sequence of live three dimensional ultrasound images not only records the activity of 
20 the interventional device but also die progress and history of the treatment procedure. 

FIGURE 15 iUustrates an embodiment in which the progress and history 
of die treatment procedure is recorded on a model of the anatomy being treated, m this 
case a wire frame model 130. This wire firame model 130 is of die heart wall of fte left 
ventricle, and may be formed by border detection of the three dimensional data set as 
25 described in US Pat 6,491.636 (Chenaletal.) or in US Pat. 5.601.084 (Sheehanetal.) 

or published European patent specification EP 0 961 135 Bl (Mumm et al). As 
treatment procedures are performed at specific points on the heart wall, fliose locations 
are detected l>y use of die sensor 32 or one of die interventional device detection 
techniques discussed above. TTiose locations 132,134.136 are tiien recorded in the 
30 proper spatial locations on die 3D wire firame model 130. where diey may appear as 
unique colors or intensities. The wire frame model may be a live, real time model 
which moves m correspondence witii die moving heart, or it may be a representation of 
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the heart intermediate the heart's shape and size at end systole and end diastole, or it 
may be a wire ftame model of the heart at its moment of greatest expansion at end 
diastole. 

FIGURE 16 illustrates a display screen 99 wWch shows cardiac 
5 information in three ways: a live three dimensional image 100 of the heart and an 
interventional device 30. a wire ftame model 130 of the chamber of the heart 
undergoing treatment, and multiple ECG traces 140 taken at points of the heart 
designated by die interventional device 30. The separate ECG traces may be labeled, 
colored, or visually designated in some other way to show correspondence with the 
10 locations at which they were acquired, which locations may also be shown by markers 
on the three dimensional ultrasonic image 100. the wke ftame model 130. or bofli. 
Such a display enables visual monitoring of the live procedure, a visual record of the 
proceduresperformed.andmeasurementdataacqmredatthesitesoffliepiocedmes. 

nOURES 17-21 are flowcharts which iBustrate several ways m which 
15 the three dimensional ultrasonic imag5 data acquired by die ultrasound system can be 

merged with spatially based data of the interventional device such as location or image 
data. In the process of FIGURE 17 acqmred 3D ultrasound data is volume rendered by 
the ultrasound system to form a 3D ultrasound image in step 202. In step 204 the 
volume rendered ultrasound image and data identi^g Hie position of the array 
20 transducer 50 or probe 10 is transmitted to the interventional system. In step 206 the 3D 
ultrasound image is converted to the ftame of reference of the data of the mterventional 
system. This may involve rescaling the coordinate data of the 3D liltrasound image to 
match the coordmate scaUng of the interventional system data. Once the ultrasound 
image data has been converted, the ultrasound image and the interventional device data 
25 are aligned (step 208) through the use of the probe and interventional device coordinate 
information and combined (step 210) to form a consolidated three dimensional image 
for display 28 containmg spatially accurate information about the interventional device 

and/or its procedure. 

In the process of FIGURE 18 there is no initial volume rendering of the 
30 3D ultrasound image data, histead. the process begins in step 212 with the scan 
conversion of the 3D ultrasound unage data to form a Cartesian referenced 3D data set. 
In step 214 the scan converted 3D data set and the probe or anay transducer position 
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infoimation is transmitted to the interventional system. In step 216 Ae 3D ultrasound 
image data is converted to the ftame of reference of the data of the interventional 
system. Again, this may be done by rescaHng the coordinate data of flie 3D ultrasomid 
image to match the coordmate scaling of Ae interventional system data. In step 218 the 
5 3D image data set and die interventional device data are combmed on the basis of Iheir 
common reference ftame. TTie merged data sets are then volume rendered by the 
interventional system in step 220 to produce a composite three dimensional ultrasonic 
image containmg spatially accurate infoimation about the interventional device and/or 
its procedure. 

10 In the process of nOURE 19 the data acquired by the interventional 

device, such as position or image data, is transmitted to the ultrasound system iii step 
230. In step 232 the interventional device data is converted to the ftame of reference of 
the ultrasound probe or transducer as by rescaling the data. In step 234 the 
interventional device data is combined with 3D scan converted ultrasonic image data. 
15 hi step 236 flie combined data is volume rendered to form a toee dimensional 
ultrasonic image containing spatially relevant mformation from the interventional 
system. The images thus produced are displayed on the ultrasound system display 18. 

In the process of FIGURE 20 volume rendered video data produced by 
the interventional system is transmitted to the ultrasound system in step 240. In step 
20 242, three dimensional ultrasound image data is rescaled and oriented to match the 
ftame of reference of the interventional system data. The three dimensional ultrasound 
unage data is then rendered in step 244 from the frame of reference or perspective that 
was used in the rendermg of the interventional system data. The interventional system 
video data and the 3D ultrasound image data, now rendered to the same frame of 
25 reference, may now be combined to foim a consolidated three dimensional image in 
step 246. 

In the process of FIGURE 21 volume rendered 3D ultrasound video data 
is transmitted to the interventional system in step 250. In step 252 interventional system 
image data is rescaled and oriented to the frame of reference of the 3D ultrasound 
30 rendering. In step 254 the mterventional system image data is rendered to the same 
frame of reference as that of the 3D ultrasound image. In step 256 the commonly 
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referenced interventional system video data and the 3D ultrasound video data are 
combined into a consolidated image. 

K will be ^parent to those skilled in tiie art that three ^pes of coordinate 
transformations betweai the Ihiee dimensional ultrasound data and flie interventional 
5 device location data are possible. The interventional device data can be transformed to 
die coordinate system of the 3D ultrasound hnage, or the 3D ultrasound image data can 
be transformed to the coordinate system of the interventional device data, or both sets of 
data can be translated to a user-chosen scale or fiume of reference. 

TTie principles of fee foregoing data combining processes may be applied 
10 in the embodiment of HGURE 22 which is a combined 3D ultrasonic imaging and 
interventional device system. In this embodiment the mterventional device 30 is 
manipulated by the guidance system 22 and operated to perform its procedure under 
control of the intervention subsystem 20. At times signaled by flie intervention 
subsystem or the ultrasound acquisition subsystem the device position measurement 
15 subsystem 24 acquires locational information of the imaging probe 10, the 
mterventional device 30. or both. Tlie 3D ulttasonic image data acquired by the probe 
10 and the ultrasound acquisition subsystem 12, and the interventional data acquired by 
the intervention subsystem, and the location data of the probe 10, the interventional 
device 30. or both, are then processed to form a consolidated 3D ultrasound image by 
20 the 3D image processor 14. The 3D image containing interventional device and/or 
procedure mformation is tiien displayed on the display 18. Ibe entire unagmg and 
interventional system is controlled by a user control panel 29. 

As used herein the terms "surgical region", "surgical guidance", and 
"surgery" encompass any medical procedure in which a medical instiranent or device is 
25 introduced mtoflie body for a medical purpose. 
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WHAT IS CLAIMED IS: 

1. Amethod of ultrasonically displaying an invasive medical device 
and the volumetric region of a body in which it is located comprising: 
5 scanning ttie volumetric region with beams transmitted by an ultrasonic 

anay transduce^ 

receiving echo signals ftom the volumetric region and ftom an invasive 

medical device located in the i^oi^ 

processing echo signals to produce a wide field of view of the votametric 

10 region; 

processing echo signals to produce a detailed view of flie portion of the 
volumetric region in which the mvasive medical device is located; and 

displaying both fbs wide field of view of the volumetric region and the 
detailed view of the portion of the vohmietric region in which the mvasive medical 
1 5 device is located on an image display. 

2. The method of Qaim 1. wherein proc^g the echo signals to 
produce a wide field of view comprises processing the echo signals to produce a wide 
field of view two dimensional image; and 

20 wherein processing the echo signals to produce a detailed view 

comprises producing a volume rendering of a portion of the vohmietric region. 

3. The method of Claim 2, wherdn displaying flother comprises 
displaying the wide field of view two dimensional image and the votame rendering in 

25 differrait areas of an image display. 

4. The method of Claim 3, further comprising designating flie 
spatial location of the volume rendering in the two dimensional image. 



30 



5. The method of Claim 2, wherein displaying further .comprises 
displaying the wide field of view two dimensional image and the volmne rendering in 
spatial alignment in a common area of an image display. 
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6. The method of Qaim 1, wherein processing the echo signals to 
produce a wide field of view comprises volume tendering the echo signals to produce a 
wide field of view three dimensional image; and 

5 wherein processing the echo signals to produce a detailed, view 

comprises producing a volume rendering of a portion of the volumetric region. 

7. The metfiod of Claim 6, wherein displaying fiirther comprises 
displaying the wide field of view diiee dimensional image and the volume rendering of 

10 the portion of the volumetric region in different areas of an image display. 

8. The method of Claim 7, finiher comprising designating the 
spatial location of the volume rendering of the portion of the volumetric re^on in the 
wide field of view three dimensional image. 

15 

9. The method of Claim 6, wherein displaying fiirth©r comprises 
displaying the wide field of view three dimensional image and the volume rendering of 
the portion of the volumetric region in spatial alignment in a common area of an image 
display. 

20 

10. The method of Claim 9, wherein displaying fiirthCT comprises 
displaying the volume rendering of the portion of the volumetric region in a separate 
enlarged or zoomed view. 



25 

11. The method of Claim 1, wherein displaying fiirther comprises 
displaying the detailed view of the portion of the volumetric region in an enlarged or 
zoomed view. 

30 12. The method of Claim .1, further comprising processing echo 

signals to produce a time-based display; and 
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wheiein displaying further comprises displaying the time-based display 
on an image display. 

13. The method of Claim 12, wherein processing echo signals to 
5 produce a time-based display fiir&er comprises processing echo signals to produce a 

spectral Doppler display, an M-mode display, or a color M-mode display. 

14. The mefliod of Claim 1, wherein scanning further comprises 
transmitting a relatively low beam density over a volumetric region, with a relatively 

10 high beam density being transmitted in a portion of the volumetric region in which an 
invasive medical device is located. 

15. The method of Claim 14, wherein processing echo signals to 
produce a wide field of view further comprises processing echo signals received ftom a 

15 low beam density region of the volumetric region; and 

wherein processing echo signals to produce a detailed view fiirther 
comprises processing echo signals received from a high beam density region of the 
volumetric region. 

20 16. An ultrasonic surgical guidance imaging system which acts to 

guide the placement or observe the operation of an invasive medical device comprising: 
an ultrasonic probe including an array transducer which steers ultrasonic 
beams over a volumetric surgical region which includes an invasive medical device; 

a transmit beamformer coupled to the array transducer which acts to 
25 control the spatial beam density of the beams transmitted by the array transducer in the 
volumetric regioiu 

a receive beamformer coupled to the array transducer and responsive to 
echo signals from array elements for the production of received scanlines in the vicinity 
of the invasive medical device and in the volumetric region at locations removed from 
30 the invasive medical device location; 
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an image processor re^onsive to the received scanlines for producing a 
wide field of view of the volumetric surgical region and a detailed view of the invasive 
medical device; and 

a display coupled to ttie image processor which displays bofli the wide 
S field of view of the volumetric surgical region and the detailed view of the invasive 
medical device. 

17. The ultrasonic surgical guidance imaging system of Claim 16, 
wlierein tfie display is operated to display both the wide field of view of tibie volumetric 

10 surgical region and the detailed view of the invasive medical device in spatial 
registration. 

18. The ultrasonic surgical guidance imaging system of Claim 16, 
wherein the transmit beamformer acts to control the spatial beam density of the beams 

15 transmitted by ttie array transducer to be different in the vicinity of the invasive medical 
device than in the volumetric region at locations removed firom the invasive medical 
device location. 

19. The ultrasonic surgical guidance imaging system of Claim 16, 
20 wherein the receive beamformer comprises a multiline receive beamfoimer. 

20. The ultrasonic surgical guidance imaging system of Claim 19, 
wherein the multiline receive beamformer is operated for the production of a different 
number of received multilines for each transmit beam in the vicinity of the invasive 

25 medical device than that produced in the volumetric region at locations removed firom 
the invasive medical device location. 
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GUIDANCE OF INVASIVE MEDICAL DEVICES BY 
WIDE VIEW THREE DIMENSIONAL ULTRASONIC IMAGING 

Abstract of the disclosuie: 

5 

A three dimensional ultrasonic diagnostic imaging system is operated to guide or 
observe the operation of an invasive medical device in three dimensions. The invasive 
medical device is shown in a detailed ultrasonic image and the balance of the 
volumetric region in which the device is located is shown in a wide field of view. The 
10 detailed and wide fields of view may be displayed separately or overlapping in spatial 
alignment on an image display. TTie wide field of view may be shown in two or toee 
dimensions. A quantified display may be shown together with the wide and detafled 
anatomical displays. The detailed view may also be shown in an enlarged or zoomed 
format 

15 
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